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EXECUTIVE  SUMMARY 

OBJECTIVE 

The  objective  was  to  evaluate  Fleet  acceptance  of  the  New 
Development  Technical  Manual  (NDTM)  and  System  Evaluation  and 
Trainer  (SEAT)  software.  These  materials  represent  significant 
improvement;^  over  presently  used  materials.  An  introduction  was 
designed  to  facilitate  immediate  use  of  the  improved  materials. 
This  introduction  included  developing  media  and  hosting  workshops 
for  individual  Fleet  units.  The  subsequent  evaluation  was 
conducted  using  a  questionnaire  that  was  mailed  to  the  Fleet 
approximately  2  months  after  the  materials  were  provided  for 
Fleet  use.  The  work  was  conducted  under  the  sponsorship  of  the 
NATO  SEASPARROW  Project  Office  (NAVSEA-06P) . 

RESULTS 

People  who  frequently  use  the  NDTM  indicate  it  would  reduce 
troubleshooting  time,  improve  accuracy,  be  useful  for  all 
technicians,  and  be  useful  for  shipboard  training. 

Over  93  percent  of  the  Fleet  personnel  who  were  introduced 
to  the  NDTM  with  the  media  (video,  booklet,  and  pamphlet)  found 
the  media  to  be  similarily  useful. 

All  of  the  SEAT  software  was  evaluated  as  easy  to  use. 

The  use  of  the  SEAT  software  was  expected  *.o  result  in  a 
decrease  in  work  time. 

RECOMMENDATIONS 

Obtain  wide  community  participation  in  the  introduction  of 
new  technology.  This  participation  will  aid  in  achieving  broad 
community  support  for  new  concepts  and  systems.  The  feedback 
from  the  participants  will  serve  to  help  ensure  that  the 
attributes  of  the  product  are  maximized. 

Use  a  proactive  orientation  for  the  introduction  of  new 
technology.  This  orientation  should  provide  for  planning  to 
allow  for  anticipated  contingencies  and  must  include  open 
communication  with  developer  and  user  participants  throughout  the 
introduction  process. 

A  survey  process  should  be  considered  for  evaluating  future 
evaluation  projects  as  it  provides  both  the  development  and  user 
communities  with  an  avenue  for  sharing  information,  identifying 
problem  issues  and  concerns,  and  for  their  resolution  in  a  timely 
manner . 


CONTENTS 


INTRODUCTION  .  1 

BACKGROUND . . . .  1 

METHODOLOGY . . . .  4 

Sample  . . . . .  4 

Questionnaire  . .  5 

Procedure  . . .  5 

RESULTS  . . .  6 

NOTH  Introduction  . .  8 

NDTM  Features  . . .  9 

Manual  Preference  and  Perceived  Benefits  of  the  NDTM  .  11 

Frequency  of  use  of  NDTM  and  Perceived  Benefits  ......  11 

Use  of  Media  and  Perceived  Benefits  of  NDTM .  18 

Participant  Comments  . 22 

DISCUSSION . 22 

CONCLUSIONS  . 24 

RESULTS  from  the  SEAT  SECTION  of  the  QUESTIONNAIRE 

SEAT  Phase  II  Software  Introduction  . .  25 

Use  of  SEAT  Phase  II  Software  . . 25 

Evaluation  of  the  SEAT  Software . . .  25 

Evaluation  of  the  15  SEAT  Phase  II  Functions  . .  26 

Participant  Comments  .  28 

DISCUSSION  and  CONCLUSIONS  .  28 

RECOMMENDATIONS  . . 28 

REFERENCES  .  29 


11 


APPENDICES 


A  The  Introduction  of  the  New  Development  Technical 

Manual:  Design  and  Material  . . . .  31 

B  New  Development  Technical  (NDTM)  and  SEAT  Phase  II 

Questionnaire  .  41 

C  Survey  Results  of  New  Development  Technical 
Manual  (NDTM)  and  SEAT  Phase  II  Questionnaire: 

Comments  about  the  NDTM  and  SEAT  Phase  II  Software  50 

FIGURES 


1  Participant  background  . 

2  Manual  Preference  and  evaluation  of  NDTM  . 

3  Use  of  functional  overview  binder  and  perceived 

benefit  ..| . 

4  Use  of  transmit  major  function  binder  and  perceived 

benefit  . . . 

5  Use  of  ship  replaceable  assembly  binder  and  perceived 

benefit  . . ^ . 

I 

6  Use  of  video  and  perceived  benefit  of  NDTM  . 

I 

7  Use  of  booklet  and  perceived  benefit  of  NDTM  . 

I 

8  Use  of  pamphlet  and  perceived  benefit  of  NDTM  . 

TABLES 

1  Distribution  of  the  NDTM/SEAT  questionnaire  . 

2  People  indicating  they  did  not  use  the 

NDTM  Introduction  media  . . 

3  Evaluation  of  NDTM  features  . 

4  Evaluation  of  NDTM  fcsatures  . 


7 

12 

15 

16 

17 

19 

20 
21 

4 

8 
9 

10 


5  Correlation  between  use  of  binder  and  perceived 


benefit  .  13 

6  Evaluation  of  the  SEAT  Phase  II  tapes .  26 


7  Evaluation  of  the  15  SEAT  Phase  II  tape  functions....  27 


iii 


V 


INTRODUCTION 


The  NATO  SEASPARROW  Project  Office  is  sponsoring  a 
development  effort  to  improve  technical  materials.  The  objective 
is  to  provide  tha  Fleet  with  materials  that  facilitate  increased 
work  efficiency.  Specifically,  the  Ordnance  Publications  (OP) 
are  being  replaced  by  the  New  Development  Technical  Manual  (NDTM) 
and  improvements  are  being  made  to  the  System  Evaluation  and 
Trainer  (SEAT^  software. 

Technical  materials  are  sometimec  developed  with  little  or 
no  interaction  with  the  intended  user.  The  materials  are 
occasionally  mailed  to  the  Fleet  without  introduction,  leaving 
the  orientation  and  installation  of  the  materials  as  tasks  that 
the  operators  must  perform.  This  routine  has  often  resulted  in 
the  subopt imal  or  nonuse  of  materials.  Such  experiences 
provided  the  impetus  to  the  current  effort,  intended  to 
facilitate  the  integration  of  the  new  NATO  materials  into  the 
Fleet. 

Media  were  developed  and  an  introduction  team  was  assembled 
and  used  to  introduce  the  NDTM  and  SEAT  software  to  the  Fleet. 
The  results  of  the  introduction  effort  and  the  Fleet  response  to 
this  process  and  to  the  enhanced  NATO  SEASPARROW  technical 
materials  are  provided. 

BACKGROUND 


The  theme  for  the  development  and  introduction  of  the  NDTM 
and  SEAT  Phase  II  software  was  to  provide  user  friendly  materials 
for  the  technician.  User  participation  was  promoted  through 
orientations  to  these  materials  prior  to  final  delivery  to  the 
Fleet  (Mecherlkoff  and  Mackie,  1970) .  These  orientations  were 
Intended  to  minimize  any  uncertainty  about  the  new  materials. 
This  was  an  educational  approach  to  the  introduction  of  change, 
in  contrast  to  a  power  approach  whereby  the  user  is  simply 
Informed  that  there  is  to  be  a  change  in  their  technical 
materials  which  would  be  mailed  in  the  near  future  (Albrecht  and 
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Ropp,  1984;  Zaltman  and  Duncan,  1977). 

For  the  NDTM,  the  educational  introduction  approach  included 
the  development  of  standardized  media,  made  available  to  the 
Fleet  for  learning  about  the  NATO  materials.  These  media  were 
developed  to  facilitate  NATO  SEASPARROW  technicians  in  quickly 
becoming  familiar  with  NDTM.  Included  were  a  video,  introduction 
booklet,  and  question  and  answer  pamphlet.  The  purpose  of  these 
media  was  to  demonstrate  to  the  Fleet  that  it  could  readily 
transfer  from  the  use  of  the  OP  to  the  NDTM.  The  challenge  of 
the  task  was  to  transfer  the  information  to  the  intended  users. 
The  objective  was  to  maintain  positive  perceptions  for  the  user, 
as  user  perceptions  are  critical  in  the  acceptance  of  innovations 
(Wylie  and  Mackie,  1982)  .  We  felt  it  important  to  show  the 
changes  were  congruent  with  the  Fleets'  perspective  on  needed 
changes.  Without  these  media,  users  could  generate  a  host  of 
misinformation  regarding  the  NDTM. 

The  NDTM  was  introduced  to  the  Fleet  through  the  Naval  Sea 
Center  Pacific  and  Atlantic  (NAVSEACENPAC  and  NAVSEACENLANT) 
Commands.  These  Commands  provide  the  Fleet  with  technical 
assistance  in  the  maintenance  of  the  equipment. 

Prior  to  introduction  and  delivery  of  the  NDTM,  a  concerted 
effort  was  made  to  transfer  knowledge  about  the  NDTM  through  user 
participation.  Included  were  usability  testing  and  a  review  by 
the  Fleet  of  a  sample  of  the  NDTM.  For  the  usability  testing, 
students  and  instructors  at  a  NATO  SEASPARROW  school  used  the 
NDTM  for  troubleshooting.  Results  show  NDTM  was  a  viable 
alternative  to  the  OP  used  for  the  maintenance  of  the  NATO 
SEASPARROW.  For  the  Fleet  review,  all  NATO  ships  received 
portions  of  the  NDTM  with  the  task  of  providing  comments  and 
questions.  This  Information  was  used  to  develop  the  question  and 
answer  pamphlet  as  well  as  inform  the  introduction  team  on  what 
issues  they  should  be  prepared  to  focus  on  when  they  introduced 
the  NDTM  to  the  Fleet. 

These  introduction  activities  provided  iterative  learning 
experiences  for  those  involved  in  the  development  and  use  of  the 
NDTM  (Argyris  and  Schon,  1972).  A  most  important  aspect  was  that 


people  at  the  development  level  received  feedback  from  users 
(Kotter  and  Schlesinger,  1979;  Mecherikoff  and  Mackie,  1970). 
The  introduction  of  new  technology  can  lead  to  or  be  accompanied 
by  an  increase  in  the  level  of  ambiguity  about  the  new 
technology.  If  there  is  no  concomitant  information  about  the  new 
technology,  people  may  create  their  own  information.  They  may 
infer  attributes  about  the  change,  consider  how  the  change  could 
impact  their  work,  and  arbitrarily  assess  the  utility  of  the 
change.  It  is  conceivable  that  change  is  impeded  by  the  intended 
user,  who  is  provided  little  or  incorrect  information  about  a 
change.  And  different  people  can  arise  as  perceived  salient 
sources  of  information  about  the  new  technology.  A  variety  of 
issues  can  surface;  for  example,  people  feeling  a  los>  of  control 
over  their  work  environment. 

In  summary,  the  introduction  of  the  NDTM  met  the  criteria 
for  minimizing  resistance  to  change  (Bennis,  Benne,  and  Chin, 
1969).  Knowledge  about  the  new  material  Was  disseminated  to  the 
Fleet;  a  source  with  whom  the  Fleet  had  trust  was  the  introducer 
of  the  change  to  the  new  material;  and  the  Fleet  was  provided 
opportunities  in  directing  the  development  of  the  NDTM.  A  model 
depicting  the  introduction  and  an  explanation  of  the  materials 
used  for  the  introduction  are  in  appendix  A. 

An  objective  of  the  SEAT  Phase  II  software  development  was 
to  deliver  to  the  Fleet  user  friendly  software  .  User  friendly 
is  a  guideline.  The  objective  is  to  provide  an  operator  software 
that  minimizes,  if  not  totally  eliminates,  any  impediments  to 
use.  The  SEAT  Phase  II  software  was  developed  and  facilitated  by 
periodic  Engineering  Working  Group  meetings  and  usability 
testing.  At  the  meetings,  user  friendly  guidelines  were 
instituted  as  well  as  human  factors  guidance  from  MIL-STD-1472D, 
and  Smith  and  Hosier  (1986) .  A  sample  of  the  software  was 
submitted  to  usability  testing  at  a  school  where  the  NATO 
SEASPARROH  technicians  are  trained  on  the  SEAT. 

The  NDTM  and  SEAT  Phase  II  questionnaire  marks  the  last 
element  in  the  sequence  of  planned  activities  for  the 


introduction  of  the  NDTM  and  SEAT  Phase  II  software.  The 
questionnaire's  purpose  was  to  provide  an  opportunity  for  the 
Fleet  to  evaluate  the  NDTM  and  SEAT  software  after  they  had  used 
these  materials  for  at  least  2  months.  The  report  provides  the 
results  from  this  questionnaire. 

METHODOLOGY 

SAMPLE 

A  total  of  335  questionnaires  were  distributed  to  the  54 
ships  with  NATO  SEASPARROW  technicians.  Included  were  148  for 
the  Pacific  Fleet  and  187  for  the  Atlantic  Fleet.  Two  sources 
were  used  in  establishing  the  number  of  questionnaires  that  were 
distributed  to  the  ships.  One  source  was  from  information 
collected  when  the  NAVSEACENs  introduced  the  NDTM  to  the  ships. 
At  that  time,  a  variety  of  questions  were  asked.  One  being  the 
number  of  technicians  assigned  to  the  NATO  SEASPARROW  workcenter. 
The  Defense  Manpower  Data  Center  was  the  other  source  of 
information  used  in  determining  the  number  of  questionnaires  to 
send  to  each  ship.  This  Center  provided  a  printout  of  the  number 
of  NATO  SEASPARROW  personnel  aboard  each  NATO  ship.  For  example, 
there  are  typically  four  or  five  technicians  assigned  to  a  DD 
type  ship.  Table  1  shows  how  the  questionnaires  were 
distributed.  Questionnaires  were  also  sent  to  the  participating 
governments  in  the  NATO  consortium. 


Table  1.  Distribution  of  the  NDTM/ SEAT  questionnaire. 


QUESTIONNAIRE 

The  questionnaire  used  for  this  research  is  in  appendix  B. 
The  questionnaire  contains  items  on  participant  background,  the 
utility  of  the  NDTM  and  SEAT  software,  and  the  introduction 
media.  All  of  the  items  were  developed  specifically  for  this 
quest ionnai re . 

The  questionnaire  was  pretested  with  NATO  SEASPARROW 
technicians  aboard  one  DO  ship.  No  changes  were  made  to  the 
questionnaire  from  the  pretest. 

PROCEDURE 

The  NDTM  and  SEAT  Phase  ll  questionnaire  was  distributed  to 
the  Fleet  in  the  fourth  quarter  of  FY  90.  Each  package  of 
questionnaires  included  a  cover  letter  from  the  NATO  SEASPARROW 
project  office  elicitng  and  encouraging  participation.  Also 
enclosed  was  a  return  envelope.  Participants  were  requested  to 
complete  and  return  the  questionnaire  within  10  days  of  receipt. 

The  purpose  in  sending  questionnaires  to  all  ships  and  all 
technicians  was  to  obtain  as  large  a  response  as  possible.  Also, 
we  felt  it  is  important  to  have  a  representative,  stratified 
response.  That  is,  one  reflecting  the  composition  of  the  Fleet: 
different  types  of  ships,  different  pay  grades,  and  different 
locations.  Typically  only  a  small  proportion  of  questionnaires 
that  are  distributed  are  ever  returned.  In  the  case  of  the 
Fleet,  with  unique  ship  scnedules  and  other  tasking,  information 
on  ships'  status  is  teruous,  such  that  a  set  time  can  be 
established  when  technicians  will  be  available  to  work  on 
completing  a  questionnaire.  So,  senaing  out  many  cfuestionnaires 
was  done  to  ensure  that  stratification  would  be  achieved. 
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RESULTS 

Eighty-seven  questionnaires  were  received  from  the  US  Navy 
and  three  from  the  NATO  participating  governments.  Figure  1 
shows  the  87  questionnaires  are  from  22  ships  of  which  61  percent 
are  DDs.  The  majority  of  respondents  are  third  class  Fire 
Control  personnel,  FC3  (55  percent)  and  the  majority  of  ships  (67 
percent)  are  from  the  Pacific  Fleet. 
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NDTM  INTRODUCTION 


The  majority  of  the  respondents  were  introduced  to  the  NDTM 
by  a  Naval  Sea  Center  (NAVSEACEN)  representative  (64  percent) . 
Most  of  the  other  respondents  learned  about  the  NDTM  from  the 
workcenter  supervisor  (15  percent).  These  results  reflected 
conditions  in  both  the  Pacific  and  Atlantic  Fleets. 

The  materials  used  for  the  introduction  were  a  video, 
introdncrion  booklet,  and  question  and  answer  pamphlet.  The 
video  was  seen  by  72  percent  of  the  participants;  the 
Introduction  booklet  and  question  and  answer  pamphlet  were  read 
by  72  and  63  percent,  respectively.  These  media  were  well 
received.  Over  93  percent  of  those  who  were  introduced  to  the 
NDTM  with  these  media  found  them  somewhat  or  very  useful. 

The  percentage  of  technicians  who  did  not  have  access  to  or 
use  these  materials  was  video,  28  percent;  booklet:  28  percent; 
and  pamphlet,  37  percent.  Table  2  shows  differences  for  these 
introduction  media  were  found  between  location  (Pacific  and 
Atlantic)  and  ship  type  [DD  and  CV(N)).  For  example,  there  were 
fewer  participants  in  the  Pacific  Fleet  compared  to  those  in  the 
Atlantic  Fleet,  who  stated  they  did  not  view  the  video. 

Table  2.  People  indicating  they  did  not  use  the  NDTM 

introduction  media. 


Booklet 

_ Pamphlet 

% 

% 

* 

Pacific 

21 

28 

40 

Atlantic 

41 

28 

31 

DD 

19 

27 

33 

CV(N) 

36 

29 

43 
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NDTM  FEATURES 


The  NDTM  consists  of  three  binders:  System  Functional 
Overview;  Transmit  Major  Function;  and  Ship  Replaceable  Assembly 
(SRA) .  The  Overview  supports  training,  theory  of  operation,  and 
introduction  to  troubleshooting.  The  Transmit  Major  Function 
supports  fault  detection,  fault  isolation,  and  repair  to  the 
shipboard  replaceable  assembly  level.  The  SRA  provides  component 
drawings  and  part  numbers.  Table  3  shows  all  of  the  binders 
were  favorably  evaluated.  The  Ship  Replaceable  Assembly  binder 
received  the  highest  evaluation  on  the  usefulness  criteria. 
Ninety-one  percent  of  the  people  who  used  this  binder  reported 
that  the  features  in  the  binder  were  useful. 

Table  3.  Evaluation  of  NDTM  features.  (The  percentage  shown  is 
the  average  for  all  features  in  a  binder.) 


Feature 

Importance 

(%)®  Accuracy 

(%)**  Usefulness  (%)' 

System  Functional 
Overview  (MMO-011) 

80 

85 

83 

System  Functional 
Overview  (MMO-012) 

78 

87 

80 

Transmit  Major 
Function  (MMO-050) 

87 

90 

87 

Ship  Replaceable 
Assembly  (MMO-llO) 

i 

92 

89 

91 

a  Important  or  very  Important . 
b  Mostly  or  completely  accurate. 
C  useful  or  very  useful. 


I  Table  4  shows  the  evaluation  of  the  specific  features  within 
each  of  the  NDTM  binders.  The  numbers  shown  in  the  table 
indijcate  the  percentage  of  people  evaluating  the  features  as 
important,  accurate,  and  useful.  All  of  the  features  in  these 
binders  were  well  received.  With  few  exceptions,  the  features 
were  evaluated  high  on  these  criteria  by  most  people. 
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Table  4.  Evaluation  of  NOTM  features. 


8v«f  m  FUnctiowit  Ov>rv}>w  MMO-OII  Impoftonc*  («)  Accuracy  (b)  U«efulrw— (c) 


Numbering  System 

72(d) 

83 

77 

System  Functional  Overview  (SFO) 

Level  1 

77 

84 

88 

System  Functional  Overview  (SFO) 

Leveia 

80 

87 

83 

System  Functional  Overview  (SFO) 

Levels 

83 

85 

82 

Operational  Flow  Diagram  (OFO)  and 

Operational  Sequence  Diagram  (OSD) 

86 

87 

85 

Svtttm  Functfonal  Ovdrvtew  MM04)12  Importanc*  Accuracy  Ua»fultwt« 


System  Level  Fault  Resolution  Matrices 

79 

66 

81 

System  Performance  Monitor  Fault  Index 

82 

89 

82 

System  Level  Indication  Fault  Directory 

80 

85 

80 

Dally  System  OperabBity  Test  (DSOT) 
and  Radar  Operabifity  Test  (ROT) 

Sequence  Diagrams 

70 

88 

76 

Tf«n«mlt  Mafor  Functfon  MMO-050  i?nDortenc»  Accuracy  U—fuln>a« 


Transmitter  Indicator  Diagram 

88 

92 

86 

Degraded  Faults  and  No  Go  Faults  Tables 

86 

91 

89 

Subfunction  Troubleshooting  Row  Charts 

87 

93 

90 

Sublunction  Signal  Row  Diagrams  (SSFD) 

84 

90 

86 

Detailed  Functional  Diagrams  (DFD) 

93 

86 

90 

Additional  Aids 

84 

87 

83 

Ship  Replaceable  Aswmbly  MMO-11fl  Importane*  Accuracy  U«>futiw»» 


Reference  Designatlon-to-Part  Number  Index 

96 

92 

95 

Illustrated  Parts  Diagrams 

92 

88 

92 

Part  Number>to-Reference  Designation  Index 

87 

88 

87 

(a)  important  or  very  important 

(b)  mostly  or  completely  accurate 

(c)  useful  or  very  useful 

(d)  %;  The  precentage  of  participants 
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MANUAL  PREFERENCI?  AND  PERCEIVED  BENEFITS  OF  THE  NDTM 

The  NDTM  was  preferred  by  72  percent  of  the  participants.  A 
comparison  of  the  NDTM  evaluation  criteria  was  made  between 
those  who  showed  a  preference  for  the  NDTM  (72  percent)  and  those 
who  showed  a  preference  for  the  OP  (28  percent) .  The  evaluation 
criteria  are  troubleshooting  time  (TS  Time) ,  increased  accuracy 
(Accuracy) ,  which  technicians  the  NDTM  benefits  (All  Techs) ,  and 
usefulness  for  shipboard  training  (Training) .  Figure  2  shows  the 
results  of  this  comparison.  Large  differences  in  the  evaluation 
of  NDTM  are  shown  between  the  two  groups.  For  example,  70 
percent  of  the  participants  who  preferred  the  NDTM  indicate  that 
use  of  the  NDTM  would  result  in  a  decreased  troubleshooting  time. 
This  response  is  in  contrast  to  those  who  prefer  the  OP,  where 
only  39  percent  indicate  that  use  of  the  NDTM  would  result  in  a 
decreased  troubleshooting  time.  As  another  example,  95  percent 
of  the  participants  who  preferred  the  NDTM,  indicate  the  NDTM 
would  be  useful  and  benefit  all  technicians  (new  and 
experienced) .  This  response  is  in  contrast  to  those  who  prefer 
the  OP,  where  50  percent  of  the  participants  indicate  the  NDTM 
would  benefit  all  technicians. 

IREQUENCY  OF  USE  OF  THE  NDTM  AND  PERCEIVED  BENEFITS 

The  perceived  benefits  of  NDTM  are  moderated  by  preference 
for  a  manual  (NDTM  or  OP) .  The  perceived  benefits  are  also 
moderated  by  frequency  of  reference  to  the  NDTM.  We  requested 
participants  to  indicate  how  often  they  had  used  the  NDTM  before 
they  received  the  questionnaire.  The  majority  of  participants 
Indicated  they  had  used  the  Functional  Overview  (MMO-11:  62 
percent;  MMO-12:  65  percent)  and  Ship  Replaceable  Assembly  (62 
percent)  less  than  four  times. 

The  Transmit  Major  Function  was  used  more  frequently  than 
the  other  binders.  This  function  was  referred  to  less  than  four 
times  by  30  percent  of  the  respondents,  7  to  9  times  by  20 
percent,  and  more  than  15  times  by  23  percent. 
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evaluation  of  NDTM. 


A  correlational  analysis  was  used  to  find  if  the  evaluation 
of  NDTM  was  related  to  the  number  of  times  the  participants  had 
referred  to  the  NDTM.  The  results  show  the  evaluation  of  the 
benefits  of  the  NDTM  is  a  function  of  the  frequency  of  using  the 
NDTM  (table  5) .  For  example,  the  correlation  between  use  of  the 
Transmit  Major  Function  binder  and  evaluation  of  the  NDTM  as 
useful  for  shipboard  training  is  -.28,  p  <  .01.  The  more 
frequently  this  binder  was  used,  the  greater  it  was  evaluated  as 
us-sful  for  shipboard  training.  The  negative  correlation  is 
generated  from  the  reversed  wording  of  the  questionnaire  items. 


Table  5.  Correlation  between  use  of  binder  and  perceived 
benefit®. 


Number  of  Times  NDTM  Referred  to 

Functional  Transmit  Major  Ship  Replaceable 
Overview  Function  Assembly 


Evaluation  of 
NDTM 


Trouble¬ 
shooting  time 
will  decrease 

Accuracy 
will  increase 


0.21  * 

0.16 


-0.19  *  -0.17 

-0.19  *  -0.35  ** 


Useful  for  -0.27  **  -0.28  **  -0.30  ** 

Shipboard  training 


The  negative  (-)  correlation  derives  from  the  scales  of  the 


items  used  in  the  analyses:  item  5  (number  of  times) 
correlated  with  items  6,  7,  and  10. 


*  r  -0.19  to  -0.21,  p  <  .05. 


**  r  >  -0.21,  p  <  .01. 


The  significance  of  the  correlational  results  in  table  5  are 
shown  as  follows.  As  reported,  30  percent  of  the  respondents 
reported  referring  to  the  Transmit  Major  Function  binder  less 
than  four  times.  Within  this  group,  50  percent  indicate  they 
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prefer  the  NDTM  for  maintenance  information  and  50  percent 
indicate  they  prefer  the  OP.  A  shift  in  preference  changed 
dramatically  toward  the  NDTM  for  the  group  of  participants  who 
referred  to  the  NDTM  more  than  15  times.  In  this  group,  80 
percent  of  the  people  indicate  they  prefer  the  NDTM  over  the  OP 
for  maintenance  information.  Figures  3,  4,  and  5  provide  the 
results  from  frequency  of  use  for  the  three  binders.  For 
example,  in  figure  3,  57  percent  of  all  participants  who  used  the 
Functional  Overview  less  than  four  times  report  that  accuracy 
will  increase.  Eighty-three  percent  of  all  the  people  who  used 
the  Functional  Overview  mere  than  15  times  expect  accuracy  to 
increase. 
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Figure  4  shows  only  55  percent  of  the  participants  who  used 
the  Transmit  Major  Function  more  than  15  times  expect  that 
troubleshooting  time  will  decrease.  This  result  is  much  lower 
than  the  results  on  expected  troubleshooting  time  using  the  other 
two  binders.  These  results  may  be  due  to  the  large  percentage 
(20  percent)  of  participants  who  expected  no  change  in 
troubleshooting  time  with  use  of  the  NDTM  Transmit  Major  Function 
binder.  This  same  pattern  was  shown  for  accuracy  where  65 
percent  of  the  people  in  the  higher  use  category  (more  than  15 
times)  expect  use  of  the  NDTM  Transmit  Major  Function  binder  to 
result  in  greater  accuracy.  No  change  in  expected  accuracy  was 
reported  by  35  percent  of  the  participants. 

The  results  were  the  same  for  people  in  the  Pacific  and 
Atlantic  Fleets.  On  all  the  criteria  shown  in  figures  3,  4,  and 
5  there  were  no  practical  differences  between  the  evaluations  by 
these  Fleets.  There  was  a  maximum  of  a  5  percent  difference. 

USE  OF  MEDIA  AND  PERCEIVED  BENEFITS  OF  NDTM 

There  were  differences  in  perceived  benefits  of  the  NDTM 
between  people  who  indicated  the  Introduction  media  were  useful 
and  those  who  had  not  used  the  media.  Figures  6,  7,  and  8  show 
the  results  of  this  analysis.  The  greatest  differences  were  for 
the  booklet  that  consisted  of  definitions  and  examples  of  the 
NDTM  features  (figure  7).  For  example,  for  the  people  who 
indicated  the  booklet  was  useful,  these  same  people  indicated 
that  the  use  of  the  NDTM  would  result  in  less  troubleshooting 
time  (86  percent) .  In  contrast,  42  percent  of  the  people  who  did 
not  use  the  booklet  reported  use  of  the  NDTM  would  result  in  less 
troubleshooting  time. 
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benefits  of  NDTM 
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perceived  benefits  of  NDTM 


PARTICIPANT  COMMENTS 


Participants  submitted  27  comments  shown  in  appendix  C.  A 
reading  of  the  NDTM  comments  shows  that  most  of  the  comments  are 
either  recommendations  or  problems  with  the  NDTM.  While  some  of 
the  problems  are  not  shared  by  others,  there  may  be  a  few 
problems  that  need  attention.  Specifically,  comments  from 
participants  64  and  81.  There  is  also  a  request  from  two 
participants  for  the  inclusion  of  the  transmitter  schematics. 

The  two  comments  about  the  schematics  are  in  contrast  to  the 
results  from  the  report  on  the  survey  used  to  capture  Fleet 
questions  after  reviewing  a  sample  of  the  NDTM  (Sander,  1988) . 
The  report  stated  that  many  of  the  CV(N)  NATO  technicians  were 
not  pleased  with  the  exclusion  of  the  schematics  in  the  ND^'M. 

DISCUSSION 

The  success  of  a  change  can  be  attributed  to  the  adequacy  of 
the  problem  identified,  the  utility  of  the  change,  and  the 
probability  of  implementing  the  change.  These  three  objectives 
have  been  accomplished.  Smillie  and  Curran  (1983)  identified  the 
specific  problems  in  the  OP  using  human  factors  criteria  and 
input  from  the  Fleet.  The  NDTM  is  viewed  by  the  Fleet  as  an 
acceptable  alternative  to  the  OP.  And  the  NDTM  was  successfully 
introduced. 

The  unique  characteristics  of  the  NDTM  necessitated  an 
effort  to  facilitate  Fleet  orientation  to  the  NDTM,  an  effort 
that  would  prompt  implementation  of  the  NDTM  by  the  Fleet.  An 
educational  approach  was  taken  through  the  development  of  media 
that  the  technicians  used  to  learn  about  the  NDTM.  This  was  an 
effort  in  managing  attention  as  well  as  providing  a  standardized 
diffusion  of  information  (Van  De  Ven,  1986;  Zaltman  and  Lin, 
1971) .  The  results  from  this  strategy  showed  that  the 
technicians  who  indicated  the  media  were  usesful  (video,  booklet, 
and  pamphlet) ,  perceived  greater  benefits  from  use  of  the  NDTM 
than  others  who  did  not  use  the  media.  Additionally, 
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NAVSEACENPAC/LANT  provided  a  trusting  link  for  emulating  the 
transfer  from  the  use  of  the  OP  to  the  NDTM. 

The  introduction  effort  accomplished  critical  goals  that  are 
essential  to  implementing  change  (Radnor,  Rubenstein,  and  Tansik, 
1970) .  The  media  raised  an  awareness  of  the  benefits  to  be 
gained  from  use  of  the  NDTM  and  facilitated  an  understanding  of 
the  NDTM.  The  Fleet  was  involved  in  the  evolution  of  the  NDTM 
through  participation  in  usability  testing  and  reviewing  a  sample 
of  the  NDTM.  Most  important  was  the  support  for  the  change  from 
the  NATO  SEASPARROW  Project  Office.  This  support  was  evident  in 
time  alloted  for  developing  the  introduction  effort  and  the 
people  made  available  for  support. 

The  orientation  process  (workshop)  may  have  been 
instrumental  in  generating  an  interest  in  and  subsequent  use  of 
NDTM.  Results  show  that  more  frequent  use  of  NDTM  was  associated 
with  greater  perceived  benefits.  Previous  research,  by  Zajonc 
(1968) ,  has  also  shown  this  relationship. 

The  significance  of  the  results  is  seen  when  other  data  are 
brought  to  light.  Before  NDTM  was  introduced  into  the  Fleet,  a 
sample  of  the  NDTM  was  mailed  to  all  NATO  SEASPARROW  workcenters. 
A  questionnaire  followed  that  included  a  question  on  position 
toward  the  NDTM.  Of  the  135  responses,  29  percent  reported  they 
were  not  sure  if  they  wanted  to  change  from  using  the  current 
manual  to  the  use  of  the  NDTM  and  11  percent  reported  they 
preferred  the  OP.  Also,  435  questions  were  submitted  (Sander, 
1988).  These  questions  were  in  three  areas:  general  questions 
about  NDTM;  organization  and  use;  and  technical.  The  question 
and  answer  pamphlet  contained  the  answers  to  these  questions. 

The  introduction  effort  was  responsive  to  the  Fleets'  need 
for  information  about  the  NDTM.  Providing  the  question  and 
answer  pamphlet,  booklet,  and  video  may  have  been  instrumental  in 
allaying  doubts  about  transferring  from  use  of  the  OP  to  use  of 
the  NDTM. 

The  correlational  results  showed  that  those  who  used  and 
benefited  from  the  NDTM  introduction  media  had  a  positive 
evaluation  of  the  NDTM.  This  may  mean  that  the  knowledge 
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evaluation  of  the  NDTM.  This  may  mean  that  the  knowledge 
transfer  concept  was  applicable.  Given  an  acceptable  new 
technology,  people  who  are  informed  about  the  benefits  of  this 
technology  are  likely  to  elect  to  use  it. 

CONCLUSIONS 

The  NDTM  is  being  positively  received  by  the  Fleet.  The 
expended  effort  in  developing  the  /.DTM  introduction  media  was 
worthwhile  as  these  materials  were  useful  to  the  Fleet. 

The  survey  process  provided  an  opportunity  for  the  Fleet  to 
submit  the  results  from  their  review  and  use  of  NDTM.  Results 
show  that  more  frequent  use  of  NDTM  resulted  in  a  more  positive 
evaluation  of  NDTM.  This  positive  relationship  supports  a 
coordinated  introduction  effort  that  facilitates  immediate  use  of 
new  technology  so  the  Fleet  can  work  towards  a  point  where  they 
can  make  a  knowledgeable  evaluation. 

The  NDTM  introduction  was  a  proactive  effort  on  the  part  of 
the  NATO  SEASPARROW  Project  Office.  They  recognized  that  the 
NDTM  was  a  significant  change  from  the  OP  (current  technical 
manual) ,  and  therefore  would  require  a  coordinated  and  planned 
effort  to  transition  the  Fleet  to  use  the  NDTM.  The  goal  was  to 
minimize  resistance  to  the  change.  This  goal  was  achieved  by 
developing  the  introduction  media  and  workshops. 
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RESULTS  FROM  THE  SEAT  SECTION  OF  THE  QUESTIONNAIRE 
SEAT  PHASE  II  SOFTWARE  INTRODUCTION 

The  SEAT  Phase  II  tapes  were  introduced,  for  the  most  part, 
by  the  NAVSEACEN  representatives  (57  percent) .  In  addition, 
introductions  were  provided  by  work  center  supervisors  (14 
percent),  and  by  some  of  the  technicians  (13  percent).  These 
results  reflect  introduction  efforts  in  both  the  Pacific  and 
Atlantic  Fleets.  There  were  some  differences  in  perceived 
usefulness  of  the  SEAT  introduction.  While  88  percent  of  the 
Atlantic  fleet  reported  that  the  introduction  was  useful,  73 
percent  of  the  Pacific  fleet  did  so. 

USE  OF  SEAT  PHASE  II  SOFTWARE 

The  participants  indicated  how  often  they  had  used  the  five 
SEAT  Phase  II  tapes  before  completing  the  questionnaire.  The 
most  frequently  used  tapes  were  the  Operational  Program  and  Off- 
Line  Maintenance.  The  Operational  Program  and  Off-Line 
Maintenance  tapes  were  used  more  than  15  times  by  82  percent  and 
66  percent,  respectively,  of  the  participants.  The  Seat 
Diagnostics  tape  was  used  less  than  four  times  by  33  percent  of 
the  participants  and  more  than  15  times  by  33  percent  of  the 
participants.  The  System  Training  and  Data  Playback  tapes  were 
used  less  frequently.  Most  of  the  participants  indicated  they 
used  these  tapes  less  than  four  times. 

EVALUATION  OF  THE  SEAT  SOFTWARE 

Each  of  the  five  SEAT  Phase  II  tapes  was  evaluated  on  ease 
of  use,  adequacy  of  information,  and  expected  change  in 
performance  from  using  a  tape.  Table  6  shows  all  software  was 
evaluated  as  easy  to  use.  Differences  were  found  between  the 
tapes  for  adequacy  of  information.  The  System  Training  and  Data 
Playback  tapes  were  evaluated  as  containing  all  or  most  of  the 


information  needed  by  69  percent  and  71  percent,  respectively,  of 
the  participants.  For  the  System  Training  tape,  20  percent 
reported  that  only  some  of  the  information  was  provided  in  the 
tape.  These  relatively  low  ratings  are  in  contrast  to  the  higher 
ratings  shown  for  the  other  three  SEAT  Phase  II  tapes. 

The  participants  view  the  use  of  all  tapes  as  decreasing 
their  work  time.  The  decrease  in  work  time  ratings  is  low  in 
comparison  to  the  ratings  in  the  other  two  categories . 


Table 

6.  Evaluation 

Of  the  SEAT  Phase  II 

tapes . 

Tape 

I 

Easy  to  Use 
(%)  a 

Adequate  Information 
(%) 

Decrease  in 
Work  Time 
(%) 

System 

Training 

82 

69 

63 

Operational 

Program 

88 

94 

65 

Off-Iiine 

Maintenance 

i 

88 

88 

63 

Data 

Playback 

80 

71 

59 

SEAT 

Diagnostics 

89 

83 

70 

a  %  »  the  percent  of  the  total  number  of  responses. 


EVALUATION  OF  THE  15  SEAT  PHASE  II  FUNCTIONS 

Aside  from  the  time  of  day  function,  all  of  the  functions 
received  favorable  evaluations  by  the  respondents.  Table  7 
provides  the  results. 
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Table  7.  Evaluation  of  the  15  SEAT  Phase  II  tape  functions. 


Function 

Importance** 

(%) 

Accuracy" 

(%) 

Usefulness" 

(%) 

Time  of  Day 

24 

74 

29 

Off-Line 

ilaintenance 

99 

95 

94 

Recorded 

Data  Display 

82 

90 

86 

Communication 

Status 

96 

94 

90 

Air  Target 

Training 

84 

86 

81 

On-Line 

Signal  Monitor 

91 

92 

94 

Data  Recording 

81 

84 

79 

On-Line  Performance 
Monitor 

93 

83 

87 

2D  Acquisition 
Evaluation 

85 

9i 

78 

3D  Acquisition 
Evaluation 

81 

90 

77 

System  Display 
Summary 

80 

91 

80 

Missile  Firing 
Report 

88 

86 

81 

NSSCOP  Inspect 
and  Change 

93 

94 

92 

MK157  Diagnostic 
Printout 

83 

86 

85 

Surface  Target 
Training 

74 

79 

73 

a  Important  or  very  Important. 

b 

Mostly  or  completely 

accurate. 

C  Useful  or  very  useful. 
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PARTICIPANT  COMMENTS 


Space  was  provided  in  the  questionnaire  for  cominents  on  the 
Seat  Phase  II  Tapes.  A  total  of  73  comments:'  vere  written.  Most 
of  the  comments  were  recommendations  for  improving  elements  of 
SEAT  other  than  those  evaluated  in  this  study.  All  of  the 
comments  are  provided  in  appendix  D. 

DISCUSSION  AND  CONCLUSIONS 

,  The  Fleet  is  very  satisfied  with  the  SEAT  Phase  II  software. 
The  time  spent  in  the  Engineering  Working  Groups  and  the  In- 
Process  Reviews  has  resulted  in  software  that  the  Fleet  readily 
accepts.  The  plethora  of  recommendations  for  further  changes  may 
be  an  indication  that  the  Fleet  perceives  there  is  an  avenue  for 
communicating  recommendations  that  will  result  in  the  changes 
they  want. 

The  results  showed  that  the  use  of  the  Phase  II  software 
would  result  in  a  decrease  in  work  time  by  a  moderate  percentage 
of  the  participants.  This  may  mean  that  regardless  of  improved 
ease  of  use  and  adequacy  of  information,  the  technicians  are 
working  as  efficiently  as  possible.  There  may  be  other 
constraints  in  the  working  environment  that  the  Phase  II  tapes 
have  not  impacted. 

RECOMMENDATIONS 

Obtain  wide  community  participation  in  the  introduction  of 
new  technology.  The  feedback  from  the  participants  will  serve  to 
ensure  that  the  attributes  of  the  product  have  been  maximized. 

Use  a  proactive  orientation  for  the  introduction  of  new 
technology.  Included  in  this  orientation  are  planning  for 
anticipated  contingencies  and  communication  with  all  participants 
in  the  process. 

The  survey  process  should  be  considered  for  future 
evaluation  projects  as  it  provides  both  the  development  and  user 
communities  with  an  avenue  for  sharing  information. 
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THE  INTRODUCTION  OF  THE  NEW  DEVELOPMENT  TECHNICAL  MANUAL 

Design  and  Material 
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This  appendix  includes  material  used  in  the  concept  and 
introduction  of  the  New  Development  Technical  Manual  (NDTM) . 
Included  are  the  model  that  guided  the  introduction  effort  and 
the  NAVSEACENPAC  booklet  used  for  instruction  on  the  elements  of 
the  effort.  The  NAVSEACENLANT  booklet  was  the  same.  Following 
are  some  comments  about  these  materials. 

MODEL 

The  model  shows  the  use  of  the  media  at  the  Schools  and 
Missile  Conferences.  Students  and  Fleet  personnel  were 
considered  as  possible  agents  for  introducing  the  NDTM  to  the 
Fleet.  If  not,  NAVSEACENPAC/LANT  were  to  be  used.  After  the 
NDTM  introduction,  the  evaluation  process  began,  concluding  with 
feedback  from  the  NSSMS  Project  Office  on  the  outcome  of  the  NDTM 
introduction  to  the  NSSMS  Schools,  conferences,  and  Fleet. 

The  NDTM  was  not  delivered  tc  the  schools  or  made  available 
at  missile  conferences  prior  to  delivery  to  the  Fleet.  Thus,  the 
Introduction,  as  shown  in  the  report,  was  mostly  done  by 
NAVSEACENPAC/LANT . 

NSSMS,  NEW  DEVELOPMENT  TECHNICAL  MANUAL,  FLEET  INTRODUCTION  BY 
NAVSEACENPAC 

This  package  was  provided  to  the  NAVSEACEN  people  at  a 
workshop.  The  purpose  of  the  workshop  was  for  training  in  the 
introduction  procedure  and  the  media. 

Included  in  the  package  was  information  on  the  results  of 
the  usability  testing,  the  media,  the  fleet  introduction 
workshop,  and  record  keeping. 


NSSMS 


September  1989 


NEW  DEVELOPMENT  TECHNICAL  MANUAL 


The  technical  manual  used  for  maintaining  the  NSSMS  wil] 
soon  be  replaced  by  the  New  Development  Technical  Manual  (NDTM) . 
NAVSEACENPAC/LANT  will  play  a  key  role  in  this  transition  bj 
personally  introducing  the  NDTM  to  the  fleet.  Introductions 
will  be  accomplished  through  workshops  to  be  held  at  a  shore 
based  location  or  aboard  ship.  The  goal  is  to  introduce  the  NDT> 
to  the  fleet  as  soon  as  possible. 

The  enclosed  information  is  intended  to  provide  backgrounc 
information  on  the  NDTM  materials  and  procedures  to  be  used  ir 
the  introduction  of  the  NDTM  to  the  fleet.  The  purpose  of  the 
materials  is  to  minimize  uncertainty  about  the  use  of  the  NDTM  as 
a  replacement  for  the  technical  manual. 

The  objective  of  a  personalized  Introduction  is  to  promote 
immediate  use  of  the  NDTM  by  the  fleet.  The  alternative  ol 
mailing  the  NDTM  would  likely  result  in  delayed  or  non-use  ol 
the  NDTM.  The  fleet  will  benefit  from  a  concerted  effort  or 
the  par  t  of  NAVSEACENPAC  to  involve  the  technicians  as  much  as 
possible  in  learning  about  the  NDTM. 

Included  in  this  package  is  background  information  on  ths 
NDTM;  results  of  the  NDaw  usability  test;  description  of  ths 
NDTM  introduction  materials;  agenda  for  the  NDTM  workshop;  anc 
workshop  attendance  form. 

POC  for  the  NDTM  fleet  introduction  is  Steve  Sander:  NOSC, 
Code  443,  San  Diego,  CA,  92152-5000.  Autovon  553-8006,  (619) 
553-8006. 
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NEW  DEVELOPMENT  TECHNICAL  MANUAL  (NDTM) 


The  NDTM  represents  a  functional  orientation  of  the  NSSMS.  The  technical 
information  is  organized  by  major  function  with  all  the  troubleshooting  information  for 
each  of  eight  functions  contained  in  their  respective  binders.  As  the  cost  of  developing 
the  entire  NDTM  was  prohibitive,  the  NATO  Seasparrow  Project  Office  (NSPO)  decided  to 
acquire  NDTM  incrementally.  The  original  intent  was  for  NDTM  to  completely  supercede 
the  existing  44  binders  for  both  7H  and  7M.  As  the  7H  systems  are  being  phased  out, 
NSPO  directed  that  any  effort  on  the  7H  version  be  stopped.  NDTM  will  cover  the  entire 
NSSMS  instead  of  the  current  System  (SW26i-T2),  two  Subsystems  (SW261-Q2  &  SW394- 
Fl),  and  three  Unit  manuals  (ST890-MO,  SW264-D2,  &  SW265-F2)  approach  in  the  existing 
TMs. 

The  first  increment  of  the  NDTM  will  consist  of  three  binders.  The  three  binders  are 
System  Functional  Overview  (SFO)  binder  (SW261-TM-MMO-011,  012),  Transmit  Major 
Function  binder  (-050),  and  the  Transmitter  Ship  Replaceable  Assembly  Identification 
binder  (-110).  The  first  increment  is  built  on  a  3UN  1986  baseline  as  the  contract  was  let 
in  OCT  1986.  A  separate  change  page  package  supporting  the  Block  IR  ORDALT  will  be 
issued  to  sites  receiving  the  ORDALT  which  will  enable  the  introduction  of  NDTM  to  all 
7M  users  with  or  without  Block  IR  installed. 

The  three  binders  of  NDTM  will  be  used  in  conjunction  with  the  existing  TM  set  with 
exception  of  the  Transmitter  FOID  binder.  This  one  FOID  binder  will  be  superceded  by 
the  new  Transmit  Major  Function  binder.  Additionally,  the  existing  Transmitter  Power 
Distribution,  Transmitter  Schematics,  and  the  Transmitter  Parts  Lists  will  be  purged  from 
SW261-Q2-MMO-000.  The  SFO  third  level  (SFO  III)  diagrams  have  been  developed  for 
each  of  the  43  subfunctions  in  NSSMS.  These  diagrams  will  reference  either  the  Transmit 
Major  Function  binder  for  troubleshooting  the  Transmit  Function  or  the  troubleshooting 
material  (FOIDs,  Connection  Diagrams,  etc.)  in  the  existing  TM  set.  The  SFO  III  diagrams 
use  as  a  method  of  accessing  the  troubleshooting  materi^  in  the  existing  TMs  has  been 
validated  as  a  useful  tool  in  troubleshooting. 

The  NDTM  utilizes  a  work  package  concept  in  that  all  of  the  material  for 
troubleshooting  a  failed  SRA  is  contained  in  a  single  binder.  There  will  be  no  need  to  use 
more  than  one  binder  to  isolate  to  an  SRA  and  find  parts  ordering  information  (p/N  and 
FSCM)  when  NDTM  is  COMPLETELY  developed.  The  Transmit  Major  Function  binder  is 
the  only  completely  developed  work  package  binder  as  part  of  the  first  increment. 

The  Detailed  Functional  Description  is  packaged  with  the  Detailed  Functional 
Diagrams  (Signal  Flow  type  figures)  in  the  Major  Function  binder,  and  the  General 
Functional  Description  is  packaged  with  the  three  levels  of  SFO  diagrams  (Block  Diagram 
overview  figures).  In  the  Transmit  Major  Function  binder,  there  is  a  set  of  Additional 
Aids  (19  total,  17  for  the  Radar  Ready  and  2  for  the  Radar  Radiate  subfunctions)  that 
presently  reside  in  the  DCAP  binder  in  the  Transmitter  Troubleshooting  Guide.  When 
other  major  function  binders  are  developed,  there  will  be  Additional  Aids  to  assist  in 
troubleshooting  difficult  faults  in  many  of  the  other  subfunctions. 


USABILITY  TESTING  OF  THE  NATO  SEASPARROW  NEW  DEVELOPMENT  TECHNICAL  MANUAL; 

MARE  ISLAND  AND  DAM  NECK 


Resistance  to  major  change  is  the  usual  rule  for  anything  new.  Acceptance  of  the 
NDTM  by  technicians,  however,  was  overwhelming,  the  majority  actively  preferring  it  to 
the  current  manual.  Virtucdly  every  feature  and  characteristic  of  the  NDTM  elicited 
favorable  comment.  Especially  liked  was  the  structure  that  permitted  all  information 
needed  to  troubleshoot  a  malfunction  to  be  found  in  one  volume.  Signal  flow  could  be 
more  easily  tracked  than  in  the  current  multivolume  manual.  The  numbering  system,  once 
it  was  mastered,  made  it  easy  to  access  whatever  narrative  or  schematic  information  was 
desired.  Locator  Diagrams  included  on  schematics  provided  specific  detail  so  that  a  part 
or  test  point  could  be  easily  found. 

The  majority  of  the  few  negative  comments  were  actually  suggestions  for  further 
improvement  of  the  manual.  Foremost  among  these  were  comments  regarding  the 
difficulty  in  learning  how  the  new  numbering  system  worked  and  the  need  for  a  clearer 
introduction  to  the  structure  and  mechanics  of  the  manuctl. 

Concern  weis  expressed  that  the  NDTM  would  be  implemented  in  increments  as 
functional  sections  are  completed.  This  would  edfect  training,  having  to  teach  and  learn 
both  current  and  new  troubleshooting  approaches,  with  a  periodically  changing  mix  of  the 
two.  It  would  also  make  it  necessary  to  use  and  maintai.i  two  partially  complete  sets  of 
maintenance  documents  aboard  ship.  Virtually  all  the  technician  participants  were 
anxious  for  the  complete  NDTM  to  be  implemented  as  soon  as  possible. 

Even  with  only  limited  familiarity  and  experience  with  the  NDTM,  technicians  could 
use  the  new  manual  easily  and  effectively  to  diagnose  system  faults.  Although  the  NDTM 
required  them  to  adapt  to  a  functional-oriented  rather  than  equipment-oriented  approach 
to  their  tasks,  they  demonstrated  little  difficulty  in  making  the  transition  and  repeatedly 
expressed  a  preference  for  the  functional  approach. 

Introductory  instruction  for  the  NDTM  will  be  necessary.  The  orientation,  structure, 
features  and  use  of  the  manual  can  readily  be  taught  in  Class  "C"  schools.  Previously 
trained  technicians  already  aboard  ship  will  not  have  the  benefit  of  such  training  in  the 
NDTM.  Lack  of  clear,  easily  comprehended  instruction  in  the  NDTM  for  shipboard 
personnel  could  negate  the  mcinual's  acceptability  and  use  by  them. 


NDTM  INTRODUCTION  MATERIALS 


Video 


The  purpose  of  the  video  is  to  provide  the  NSSMS  coitonunity 
with  a  standardized  orientation  to  the  NDTM.  The  video  provides 
the  rationale,  attributes,  and  benefits  of  the  NDTM.  It  is  not 
for  training. 

The  video  will  be  sbown  at  each  workshop  and  a  copy  of  the 
video  will  be  given  to  each  ship's  workcenter. 


Question  and  Answer  Pamphlet 

A  sample  of  the  NDTM  was  sent  to  the  fleet  for  review  and 
comment.  This  pamphlet  contains  answers  to  the  fleet  questions 
as  well  as  a  brief  description  of  the  NDTM.  The  pamphlet  is  a 
means  for  ensuring  that  everyone  is  informed  about  the  NDTM. 
Each  workcenter  can  have  as  many  copies  of  the  Pamphlet  as  they 
want. 

Booklet 

The  booklet  contains  topics  on  the  features  of  the  NDTM  and 
sample  troubleshooting  problems.  Most  of  the  workshop  will 
center  on  the  information  in  this  booklet.  Technicians  will  be 
ready  to  use  the  NDTM  when  they  have  learned  this  information. 
Provide  each  workcenter  with  as  many  copies  as  they  want. 

Messages 

A  message  will  be  sent  from  the  NATO  SEASPARROW  Project 
Office  to  NSSMS  ships.  The  message  will  announce  the 
introduction  of  the  NDTM  to  the  fleet  by  NAVSEACENPAC.  The 
message  will  include  a  brief  description  of  the  workshop, 
invitees,  and  a  rationale  for  attending.  This  message  will  be 
referenced  when  NAVSEACENPAC  sends  a  message  to  each  ship  to 
announce  a  workshop. 
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WORKSHOP 

Each  NSSMS  ship  will  be  introduced  to  the  NDTM  at  a 
workshop.  A  workshop  will  be  conducted  at  a  shore  based  location 
or  aboard  ship.  The  shore  based  workshop  will  likely  be  used 
when  personnel  from  more  than  one  ship  are  available  for  an 
introduction.  It  is  suggested  that  one  person  from 
NAVSEACENPAC  conduct  an  introduction  for  no  more  than  eight 
people.  THE  SCHEDULING  OF  WORKSHOPS  WILL  BE  A  NAVSEACENPAC 
RESPONSIBILITY. 

The  recommended  agenda  for  the  workshop  is  shown  below. 

Welcome  to  the  New  Development  Technical  Manual  workshop.  The  purpose 
of  this  workshop  is  to  provide  you  with  an  orientation  to  the  NDTM  as  well  as 
some  experience  in  troubleshooting  with  the  NOTH.  When  you  have  completed 
this  workshop  you  will  be  prepared  to  use  the  MOTH  at  your  first  opportunity. 

o  Provide  the  background  on  the  change  to  the  NDTM  as  well  as  results 
from  the  usability  testing. 

o  Briefly  review  the  use  of  the  NDTM  introduction  materials: 

Video 

Question  and  Answer  pamphlet 

Booklet  of  NDTM  topics  and  sample  troubleshooting  problems 

o  Show  the  video 

o  Go  through  the  Booklet 


Each  NSSMS  workcenter  represented  at  a  workshop  will  receive 
the  NDTM,  TMIN  INTEGRATION,  AND,  IF  APPLICABLE,  A  BLOCK  IR 
CHANGE  PACKAGE. 

Keep  a  log  of  questions  and  concerns  expressed  at  each 
workshop.  This  information  will  be  sent  to  Steve  Sander  and 
shared  with  NAVSEACENLANT. 

Inform  the  technicians  that  they  will  receive  a  NDTM 
evaluation  questionnaire  in  approximately  six  weeks  and  are 
encouraged  to  return  it. 

Ensure  that  a  NSSMS  Workcenter  Information  Form  is  completed 
for  each  workcenter  that  is  introduced  to  the  NDTM.  Send  these 
forms  to  Steve  Sander. 

How  will  technicians  who  cannot  attend  a  workshop  be  introduced 
to  the  NDTM? 

The  NDTM  introduction  will  become  the  responsibility  of 
those  who  attended  the  workshop.  It  is  important  to  convey  to 
these  technicians  that  the  same  materials  and  procedures 
should  be  used  for  the  workshop  that  they  will  present.  It  is 
possible  that  the  technicians  could  attend  a  future  workshop. 
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NDTK  Introduction  Workshop 
NSSMS  Workcenter  Information  Form 


The  information  that  you  provide  on  this  form  will  be  used 
to  keep  track  of  ships  that  have  been  introduced  to  the  NDTM. 
Please  use  one  form  for  each  workcenter. 


SHIP ; _ _ 

Number  of  NSSMS  workcenters: _ 

Home  port; _ _ 

The  following  people  attended  the  NOTH  introduction  workshop: 


How  many  technicians  are  assigned  to  this  workcenter? _ ^ 

I 

The  workshop  was  held:  aboard  ship _  shore  based: 

Date  of  the  workshop :_ _ _ 


APPENDIX  B 

NEW  DEVELOPMENT  TECHNICAL  MANUAL  (NDTM)  AND  SEAT  PHASE  II 

questionnaire 


NEW  DEVELOPMENT  TECHNICAL  MANUAL  (NDTM)  AND  SEAT  PHASE  II 

QUESTIONNAIRE 


Recently  your  ship  received  the  NOTM  and  SEAT  Phase  II  tapes.  The  NATO 
SEASPARROW  Project  Office  is  interested  in  receiving  your  opinions  and  comments  on 
how  well  these  materials  are  working  for  you. 

If  you  have  not  used  the  NOTM  or  SEAT  Phase  II  tapes,  please  take  time  to 
familiarize  yourself  with  these  materiais  before  responding  to  the  questionnaire  items  on 
the  following  pages. 

Please  respond  to  all  of  the  questionnaire  items  and  write  any  comments  you  may 
have.  Return  the  completed  questionnaire  to  your  NSSMS  work  center  supervisor  within  7 
days.  He  will  return  all  questionnaires  to  the  Naval  Ocean  Systems  Center. 

Your  participation  is  greatly  appreciated.  The  results  from  the  survey  will  be 
published  in  a  future  NATO  SEASPARROW  newsletter.  If  you  have  any  questions,  please 
contact  Dr.  Steve  Sander  at  the  Naval  Ocean  Systems  Center,  Code  443,  San  Diego,  CA 
92152-5000:  AUTOVON  553-8006;  Commercial  (619)  553-8006 


PRIVACY  ACT  STATEMENT 

Your  participation  is  voluntary.  The  responses  you  provide  will  be  combined  with 
those  from  the  other  NATO  SEASPARROW  technicians.  Your  ship  will  not  be  identified  in 
the  results  of  this  survey. 
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NEW  DEVELOPMENT  TECHNICAL  MANUAL  (NDTM) 

Bachoiaund 

Ra*e: _ 

Ship: _ 

'C  School  graduation  date: _ 

When  did  you  first  start  using  the  NDTM: _ 

When  did  you  first  start  using  the  SEAT  Phase  II  tapes: _ 


For  the  first  item,  circle  the  number  of  any  options 

1 .  The  NDTM  was  introduced  to  me  b^ : 

that  apply. 

1 

2 

3 

4 

5 

6 

NAVSEACEN 

Representative 

'C  School 
Instructor 

Work  Center 
Supervisor 

NSSMS 

Co^rker 

Myself 
(1  learned 
how  to  use 
NDTM  on 
my  own) 

Never 

Introduced 

tome 

A  video  tape,  Introduction  Booklet,  and  Question  and  Answer  Pamphlet  were 
available  for  the  introduction  of  the  NDTM.  For  items  2, 3,  and  4,  circle  the  number  of  the 
option  that  best  describes  how  usefui  these  materials  were  for  you. 


1 

2 

3 

4 

Very  useful 

Somewhat  useful 

Not  useful 

Cannot  Evaluate 

Video 

Provided  an 

Provided  some 

The  video  was 

1  have  not  seen 

overview  of  the 

NDTM  features 
and  showed  how 

1  was  an  improve¬ 
ment  over  the 
current  OP. 

information 
about  the  NDTM. 

not  useful  for 
introducing 
the  NDTM  to  me. 

the  videotape. 

• 

1 

2 

3 

4 

Verv  useful 

Somewhat  useful 

MoLuselul 

Cannot  Evaluate 

:  Introduction 

Provided  many 

Provided  some 

ITie  Booklet  was 

1  have  not  read 

Booklet 

usefui  descrip¬ 
tions  and  examples 
on  how  to  use  the 
features  of  the  NDTM. 

useful  descrip- 
ttons  and  examples 
on  how  to  use  the 
features  of  the  NDTM. 

not  useful  for 
Introducing 
the  NDTM  to  me. 

the  booklet. 

1 

2 

3 

4 

Verv  useful 

Somewhat  useful 

Not  useful 

Cannot  Evaluate 

t  Question 

Provided  many 

Provided  some 

The  Question 

I  have  not  read 

and 

helpful  answers 

helpful  answers 

and  Answer 

the  phamplet. 

Answer 

to  general  and 

to  general  and 

Pamphlet  was 

Pamphlet 

technical  questions 
atx)ut  the  NDTM. 

technical  questions 
about  the  NDTM. 

not  useful  for 
introducing 
the  NDTM  to  me. 
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For  items  5(a}  through  5(d),  circle  one  option  that  reflects  your  experience  with  the 
NDTM. 

5.  Since  the  NDTM  was  introduced  to  you,  how  many  times  have  you  referred  to  the 
following  binders? 

Less  than  More  than 


(a)  Functional  Overview 
SW261-TM-MMO^)11 

4 

4-6 

7-9 

10-12 

13-15 

15 

(b)  Functional  Overview 
SW261-TM-MMO-012 

Less  than 

4 

4-6 

7*9 

10-12 

13-15 

More  than 
15 

(c)  Transmit  Major  Function 
SW261-TM-MMa050 

Less  than 

4 

4-6 

7-9 

10-12 

13-15 

More  than 
15 

(d)  Ship  Replaceable 

Less  than 

4 

4*6 

7-9 

10-12 

13-15 

More  than 
15 

Assembly  Identification 
SW261-TM-MMO-110 

6.  Troubleshooting  time  when  using  the  NDTM,  in  comparison  to  the  current  OP,  will: 

1  2  3  4  S 

Decrease  Decrease  Remain  Increase  Increase 

giotfiiy  somewhat  the  same  somewhat  greatly 

7.  The  accuracy  of  my  work  when  using  NDTM,  in  comparison  to  using  the  current  OP, 
will: 

1  2  3  4  5 

Decrease  Decrease  Remain  Increase  increase 

greatly  somewhat  the  same  somewhat  greatly 

8.  As  a  source  of  maintenance  information  for  the  NSSMS  transmit  function  ,  I  prefer  to 
use: 

1  2  . 

New  Devetopment  Current  OP 

Technical  Manual 

9.  NDTM  is  beneficial  to: 


1  2 
AH  technicians  New  technicians 

only 


3 

Experienced 

technicians 

only 


10.  For  shipboard  training,  NDTM  is: 


4 

Not  beneficial 
to  new  or 
experienced 
technicians 


1  2  3  4  5 

Very  useful  Somewhat  useful  Borderline  Of  little  use  Of  no  use 
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11.  Some  of  the  features  in  the  NDTM  are  listed  below  for  your  evaluation.  Indicate  your  opinion 
of  each  feature  by  assigning  a  number  from  each  of  the  three  evaluation  categories. 

EyALUAT10ll£ATEfi0RIES 


Imaoftanea 

Acetitacy 

Uaafulnaaa 

Vary  important 

1 

Compiataly  accurate 

1 

Vary  useful 

1 

Important 

2 

Moatlyaocurata 

2 

Useful 

2 

FEATURE 

Bogina 

3 

Borriatlina 

3 

Bordarlina 

3 

Not  Important 

4 

Moatly  inaocurata 

4 

Of  soma  use 

4 

Svstam  Furtetional  Ovarviaw 

MM04)11  Varyunlrnportam 

S 

Compiataly  inaocurata 

5 

Of  no  use 

S 

Numbering  System 

System  Functional  Overview  (SFO) 

Level  1 

System  Funedonai  Overview  (SFO) 

Laval2 

System  Functional  Overview  (SFO) 

Levels 

Operational  Flow  Diagram  (OFD)  and 

Operational  Sequence  Diagram  (OSD) 

! 

Sytww  Rinctloiwl  OvwviMif  MM<»012 _ Importinee _ Accuracy _ U— fulnc— 


System  Laval  Fault  Resolution  Matrices 

System  Parformanoa  Monitor  Fault  Index 

System  Level  hdeadon  Fault  Directory 

Daily  System  Operabllty  Test  (DSOT) 
and  Radar  OperablRty  Teat  (ROT) 

Sequence  Diagrams 

Truwnlt  Major  Function _ MM0050 _ Importwicc _  Accuracy _ UtaMnasa 


Transmitter  Indteator  Diagram 

Degraded  Faults  and  No  Go  Faulu  TaUee 

SublUncdon  Troublesfwoting  Flow  Charts 

Subftmclion  Signal  Flow  Diagrams  (SSFD) 

Detailed  Functional  Diagrams  (DFD) 

AddWonal  Aids 

SWpRaplaeaablaAaaainbly  MMO-110  Imoortanca  Accuracy  Uaafulnaaa 


Reference  Deslgnatlon>lo-Part  Number  Index 

Huefealad  Parts  C^agrama 

Part  Number^o-Referance  Deslgnatlan  Index 

12.  What  changes  would  you  recommend  to  improve  the  NOTM?  (Use  the  backside  of  this  page  as 
needed). 
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V. 


T.  "■/  V  r 

■  '^y 


SEAT  PHASE  II 

For  the  items  below,  circle  one  option  that  reflects  your  experience  with  the  SEAT  tapes. 

1.  Since  the  Phase  II  tapes  were  delivered  to  your  ship,  how  many  times  have  you  used  each  tape? 


/  :V' 


System  Training 

Less  than 

4 

4-6 

7-9 

10-12 

13-15 

More  than 
15 

Operational  Program 

Less  than 

4 

4-6 

7-9 

10-12 

13-15 

More  than 
15 

Off-Line  Maintenance 

Less  than 

4 

4-6 

7-9 

10-12 

13-15 

More  than 
15 

Data  Playback 

Less  than 

4 

4-6 

7-9 

10-12 

13-15 

More  than 
15 

Seat  Diagnostics 

Less  than 

4 

4-6 

7-9 

10-12 

13-15 

More  than 
15 

2.  The  System  Training  tape  is: 

1  2 

Very  easy  Fairly 

to  use  easy 

to  use 


Borderline  Fairly 

difficult 


to  use  to  use 

3.  How  much  information  does  the  System  Training  tape  provide  you? 


All  that 
I  need 


2 

Most  of 
what 
I  need 


3 

Some  of 
what 
I  need 


4 

A  little 
of  what 
I  need 


5 

Very 

difficult 
to  use 


5 

None  of 
what 
I  need 


,  /  . 


...  . 
S 


4._ Compared  to  the  old  tape,  the  time  to  complete  my  work  with  the  new  -System  Training  tape 
_ will: 


Oeaease 

greatly 


2 

Decrease 

somewhat 


5.  The  Operational  Program  tape  is: 


Remain  Increase 

the  same  somewhat 


5 

Increase 

greatly 


Very  easy 
to  use 


Fairly  easy  Borderline  Fairly 


to  use 


difficult 
to  use 


6.  How  much  information  does  the  Operational  Program  tape  provide  you? 


All  that 
I  need 


2 

Most  of 
what 
I  need 


3 

Sr'-'o  of 

W' 

I  need 


4 

A  little 
of  what 
I  need 


5 

Very 

difficult 
to  use 


5 

None  of 
what 
I  need 
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Compared  to  the  old  tape,  the  time  to 

complete  my 

work  with  the  new  Operational  Program  tape 

will; 

1 

2 

3 

4 

5 

Decrease 

Decrease 

Remain 

Increase 

Increase 

greatly 

somewhat 

the  same 

somewhat 

greatly 

The  Off-Line  Maintenance  tape  is: 

1 

2 

3 

4 

5 

Very  easy 

Fairly 

Borderline 

Fairly 

Very 

to  use 

easy 

difficult 

difficult 

to  use 

to  use 

to  use 

How  much  Information  does  the  Off-Line  Maintenance  tape  provide  you? 

1 

2 

3 

4 

5 

All  that 

Most  of 

Some  of 

A  little 

None  of 

I  need 

what 

what 

of  what 

what 

1  need 

1  need 

1  need 

1  need 

Compared  to  the  old  tape,  the  time  to  complete  my  work  with  the  new  Off'Line  Maintenance  tape 


will: 

1 

2 

3 

4 

5 

Decrease 

Decrease 

Remain 

Increase 

Increase 

greatly 

somewhat 

the  same 

somewhat 

greatly 

The  Data  Playback  tape  is: 

1  2 

3 

4 

5 

Very  easy 

Fairly  easy 

Borderline 

Fairly 

Very 

to  use 

to  use 

difficult 

difficult 

to  use 

How  much  information  does  the  Data  Playback  tape  provide  for  you? 

to  use 

1 

2 

3 

4 

5 

All  that 

Most  of 

Some  of 

A  little 

None  of 

Ifieed 

what 

what 

of  what 

what 

1  need 

1  need 

1  need 

1  need 

Compared  to  the  old 

tape,  the  time  to  complete  my  work  with  the  new  Data  Playback  tape 

will: 

1 

2 

3 

4 

S 

Deaease 

Decrease 

Remain 

Increase 

Increase 

greatly 

somewhat 

the  same 

somewhat 

greatly 
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14. 

The  Seat  Diagnostics  tape  is: 

1 

2 

3 

4 

5 

Very  easy 
to  use 

Fairly 
easy 
to  use 

Borderline 

Fairly 
difficult 
to  use 

Very 

difficult 
to  use 

15. 

How  much  information 

does  the  Seat 

Diagnostics  tape 

provide  you? 

1 

2 

3 

4 

5 

All  that 

Most  Of 

Some  of 

A  little 

None  of 

1  need 

what 

1  need 

what 

1  need 

of  what 

1  need 

what 

1  need 

16. 

Compared  with  the  old 
will: 

tape,  the  time 

to  complete  my  work  with  the 

new  Seat  Diagnostics  tape 

1 

2 

3 

4 

5 

Decrease 

Decrease 

Remain 

Increase 

Increase 

greatly 

somewhat 

the  same 

somewhat 

greatly 

17.  The  functions  of  the  SEAT  Phase  II  tapes  are  listed  here  for  your  evaluation.  Indicate  your 
opinion  of  each  function  by  assigning  a  number  from  each  of  the  three  evaluation  categories. 


U8»fuln»8« 

Very  useful  l 
Useful  2 

Borderline  3 
Of  scmeuse  4 
Ofnouse  5 


1.  Tim*  of  Day 

2.  Off-Llna  Mainlanartca 

3.  Raoordad  Data  Display 

4.  Communiealion  Status 

5.  Air  Targat  Training 

6.  On-Una  Signal  Monitor 

7.  Data  Racording 

8.  On-Una  Parformanoa  Monilor 

9.  2D  Acquisition  Evahialion 

10.  3D  Acquisition  Evaluation 

11.  Sysiam  Display  Summary 

12.  MlasS*  Firing  Raporl 

13.  NSSCOPInipaet  and  Chang* 

14.  MK157  Diagnoatic  Printout 

15.  Surtac*  Targat  Training 

EVALUATION  ..CATECiORIES 

ImaBCianca  Accurecv 

Very  important  1  Completely  accurate  1 

Important  2  Mostiy  accurate  2 

Borderline  3  Borderline  3 

Not  Important  4  Mostly  Inaccurate  4 

Vary  unimportant  5  Comi^etely  inaccurate  5 


f 
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18.  Since  the  SEAT  User's  Handbook  (ST890-MO-HBK-010:  Revision  1)  was  delivered  to  your  ship, 
how  many  times  have  you  referred  to  it? 


Less  than  ^  ^ 

4  4-6 


7-9  10-12 


13-15 


More  than 
15 


19.  The  SEAT  Phase  11  tapes  were  introduced  to  me  by; 


1 


2  3 


4  5  6 


NAVSEACEN  'C  School 
Representative  Instructor 


Work  Center 
Supervisor 


20.  The  SEAT  Phase  II  introduction  was: 


NSSMS 

Myself 

Never 

Co-worker 

(1  learned 

introduced 

how  to  use 

to  me 

the  tapes 

on  my  own) 

1 

Very 

useful 


2  3 

Somewhat  Borderline 
useful 


4  5 

Of  little  Of  no 

use  use 


21.  In  my  workcenter,  changes  are  made  or  suggested  before  problems  occur: 

1  2  3  4  5  6  7 

Always  Usually  Frequently  Sometimes  Seldom  Never  No  opinion 


22.  What  would  you  like  to  see  in  future  SEAT  upgrades? 


23.  Please  write  any  comments  about  the  Phase  II  tapes.  (Use  the  backside  of  the  page  as  needed.) 
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APPENDIX  C 

SURVEY  RESULTS  OF  NEW  DEVELOPMENT  TECHNICAL  MANUAL  (NDTM)  AND 

SEAT  PHASE  II  QUESTIONNAIRE: 

COHunents  about  the  NDTM  and  SEAT  Phase  II  Software 
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SURVEY  RESULTS  OF  NEW  DEVELOPMENT  TECHNICAL  MANUAL 
(NDTM)  AND  SEAT  PHASE  11  QUESTIONNAIRE: 

(‘See  note  below) 

QUESTION  #12  -  COMMENTS  ABOUT  THE  NDTM: 

#2:  a.  Training  taoe  does  not  work. 

b.  Make  aii  the  pubs  avaiiabie  now. 

#7:  Front  of  every  NDTM  shouid  have  a  sheet  for  description  of  number 

locating,  or  number  referencing,  i.e.  to  find  reference/figure 

number  remember  that  first  number  represents _ and  second 

number  means _ ,  or  a  reminder  to  that  affect 

* 

#8:  Use  in  conjunction  with  existing  pubs.  For  me,  existing  pubs 

seem  to  be  more  detailed,  however  NDTM  takes  you  through 
quicker. 

#9:  Used  with  old  pubs. 

#19:  Include  all  tables  and  information  from  SW26 1  -Q2-MMO'080/ 

OP4005,  as  is  not  enough  about  SDP.  Cassette  loading 
troubleshooting  needs  to  be  added. 

#21:  Try  to  make  the  whole  system  into  this. 

#24:  Have  SRA  book  (MMO- 110)  list  more  detailed  information  for  the 

entire  system;  not  Just  for  the  transmitter. 

#25:  I  want  my  schematics  diagrams  returned;  also  parts  breakdown. 

#33:  Use  more  of  the  reference  diagrams  from  the  old  pubs. 

#42:  More  function  description  on  program  itself. 

#45:  Highlight  high  fail  areas  as  per  DCAP/FBR  inputs  from  the 

Fleet/shore  commands.  Identify  nonstocked  parts  for  open 
purchase  purposes.  Plasp'c  cover  shields,  EMI  gaskets/rubber 
gaskets  on  door  assembly,  etc. 

Note:  the  number  (e.g.  42)  is  the  questionnaire  from  which  the  comments 

originate  . 
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046:  Items  #9  -  2D  Acquisition  Evaluation  and  #1 0  -  3D  Acquisition 

Evaluation: 

Need  better  ref.  to  pubs  and  circuits  involved  for  repair  and 
troubleshooting. 

Item  #1 4  -  MK1 57  Diagnostic  Printout: 

No  documentation  to  support  the  test  or  the  evaluation  of  the 
results  for  system  troubleshooting  and  repair. 

Snapshot  is  useless  and  is  not  supported  for  troubleshooting  or 
system  status  by  pubs  I 

047:  I  believe  a  complete  change  over  of  M  pubs  would  be  more 

effective  as  opposed  to  the  partial  turnover. 

048:  I  would  not  recommend  any  changes  to  the  NDTM,  I  would  however, 

run  another  questionnaire  after  the  Fleet  has  utilized  them  for  a 
longer  period  of  time. 

053:  NDTM  is  easy  to  use  but  in  some  cases,  when  they  refer  you  to  the 

current  OP  the  figure  has  already  been  deleted. 

054:  I  would  suggest  the  inclusion  of  the  schematic  diagrams  from  the 

XMTR  (old)  pubs  that  have  been  eliminated.  These  are  the  ones  for 
u  ..  individual  circuit  cards. 

055:  Include  more  detailed  functional  diagrams  to  further  breakdown 

transmit  functions. 

056:  Make  up  schematics  for  block  IR  or  ORDAL  T  items. 

058:  Some  minor  errors  with  numbering  and  small  mistakes  of  this 

kind.  Outstanding  in  all  other  respects  and  highly  advantageous  to 
technicians. 

063:  Rewrite  entire  pub  system  so  I  can  understand  them. 

064:  Found  the  ARC  reference  fault  circuits  very  difficult  to  follow 

(actually  is  a  red  major  fault,  yet  pub  indicates  yellow).  Only 
really  new  circuit  in  BLK  IR  so  a  better  writeup  would  be  useful 
for  troubleshooting  the  fault  circuit  itself. 
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#77;  Stick  to  major  function  pubs  except  in  the  schools.  The  MFP's  are 
most  used. 


080:  I  would  like  to  see  a  section  that  covers  each  control  and 

indicator,  especially  consoles,  that  describes  specific  uses  and 
reasons  the  system  would  automatically  use  that  function  or 
indication.  Do  not  necessarily  need  specific  signal  flow. 

081:  The  transmit  major  function  pub  has  made  my  job  much  easier. 

Although  some  of  the  voltage  readings  in  the  DFD  are  inaccurate 
and  should  be  reevaluated.  Those  values  may  reflect 
manufacturer  spec's  but  do  not  reflect  actual  values  in  a  live 
seasparrow  environment.  Most  of  the  waveforms  that  I  had  to 
compare  were  accurate. 

084:  I  would  get  rid  of  the  whole  set.  The  diagrams  are  confusing  and 

can  lead  into  dead  ends.  There's  nobody  I've  talked  to  who  has 
liked  them  and  this  is  probably  the  general  feeling  in  the  Fleet. 

This  should  not  be  forced  on  the  techs  like  you're  determined  to 
do. 

086:  I  have  no  recommendations  for  changes.  My  view  of  these  pubs  is 

limited  because  I  learned  the  system  with  the  old  pubs  and  am  yet 
to  become  comfortable  and  proficient  with  the  new  ones. 

087:  The  only  change  I  can  recommend  is  to  make  construction  of  pubs 

more  durable.  V^e  have  used  them  so  much  that  they  are  already 
tearing  up  vice  the  old  ops  which  have  been  used  10  times  as 
much  over  the  years  and  are  still  in  good  shape.  We  really  like 
the  way  everything  is  combined  and  right  there  in  one  pub. 
Extremely  easy  to  use  once  you  get  past  the  "old  dog  can't  learn 
new  tricks "  attitude.  We  have  found  no  mistakes  as  yet  and  find 
the  additional  aids  section  a  great  help. 


QUESTION  017: 

052:  Item  1  -  Time  of  day  -  Accuracy: 

Time  of  day  does  not  update  while  in  menu. 
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QUESTION  #22  ~  COMMENTS  ABOUT  THE  SEAT  PHASE  II 
SOFTWARE: 

#5:  Missile  check  simulated  UT  checks. 

#6:  A  better  condition  on  4821  M-4R  thru  M-9R,  and  make  them 

easier.  > 

#7:  Still  under  use  for  full  capabilities.  • 

#1 1:  An  EA  T  for  all  cabinets  that  are  tied  in  with  the  computer  instead 

of  just  RSC  and  FOC. 

#13:  More  T/S  abilities  from  SEA  T  for  all  pieces  of  equipment. 

#14:  OP  memory/auto  recovery  like  on  MK 157  comp. 

#15:  .  Auto  recovery  for  memory;  hardwired  OP  prog. 

#19:  I  would  like  to  see  the  SEA  T  upgrade  so  that  it  will  replace  the 

worthless  MK1 57. 

#23:  Basically  very  accurate  in  commonly  used  areas.  OLM  is  great 

#24:  LCHR  troubleshooting  tapes. 

#26:  Would  like  to  see  some  way  of  making  a  hard  copy  of  the  SEA  T 

display.  Also,  clearer  instructions  on  the  2D,  3D  Acq.  program. 

#29:  Built-in  system  diagnostic  testing. 

#30:  Would  like  to  see  the  OP  program  for  computer  included  in  SEA  T. 

* 

#33:  More  training  options. 

#34:  More  help  with  Fleet  introduction. 

#35:  Troubleshoot  system;  especially  XMITTER. 

#38:  Most  problems  that  I  have  encountered  are  with  the  lack  of 

knowledge  on  usage  of  the  tapes.  More  time  needs  to  be  e 
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mphasized  on  the  proper  usage  and  functions  of  these  tapes  in 
school.  They  can  be  very  beneficial  to  a  technician. 

fmO:  Giving  the  option  to  run  RAPID  test  on  the  OLM  tape  instead  of 

having  to  go  through  RAPID  just  to  do  another  test.  I  still  use  the 
old  maintenance  tape  if  I'm  in  a  hurry  to  run  another  test. 


#41:  I  would  like  to  see  more  training  for  us  on  the  tapes  and  new 

pubs.  They  (Reps)  only  gave  us  4  pubs). 

#42:  Detailed  description  of  particular  function. 

#44:  Possible  SDC  cards  that  are  faulted  printed  on  screen. 

#45:  Incorporate  a  SEA  T  keyboard  to  unlock  all  SEA  T programming  and 

troubleshooting  capabilities. 

#46:  A  full-size  keyboard  for  opening  the  complete  abilities  of  the 

SEAT  for  programming  and  troubleshooting.  A  3  1/2  disk  drive  to 
replace  the  RCRs  used  in  the  NSSMS  system  to  increase  data 
storage,  reliability,  and  increase  parts  availability  to  component 
level  through  Naval  Supply  System  and  civilian  retail/wholesale 
outlets.  It  would  also  reduce  the  amount  of  tapes  and  reduce 
storage  area. 

#47:  Full  keyboard  to  enhance  troubleshooting  techniques. 

#49:  ECM  on  SEA  T  training  scenario. 

#54:  The  ability  to  load  off-line  without  having  to  run  RAPID.  I  realize 

that  there  is  a  way  to  bypass  RAPID,  but  having  to  page  through 
assorted  menus  to  get  to  needed  screens. 

#55:  More  detail  in  troubleshooting  SEA  T. 

#56:  Printout  capabilities  of  all  screens.  More  detailed  maintenance 

documentation  on  operation  of  the  SEAT. 

#58:  A  more  comprehensive  introduction  and  explanation  of  the 

capabilities  and  limitations  as  well  as  the  use  of  the  software. 

#59:  Installation  of  keyboards. 
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4f62:  A  way  to  zero  synchros  in  the  launcher  and  director. 


More  system  status  information  with  easier  access  (examples: 

Power  Out  (M.O.  and  P.A.),  SSBG  Freqs.,  SFC  Pwr  Outputs  (current 
and  voltage),  RCVR  BITE  values).  With  no  long  explanations  of 
what  they  should  be  or  how  they  are  used.  Long  explanations  are 
usually  of  little  use  to  experienced  technicians  and  should  be 
saved  for  separate  training  periods. 

1t65:  To  be  able  to  completely  control  the  entire  system,  from  the  SEA  T. 

if66:  Fewer  tapes. 

^4:  A  separate  "EAT  tape. 

W7S:  More  test  capabilities. 

if76:  Intergrate  the  computer  OP  program  on  the  same  tape  as  the 

off-line  maintenance. 

Air  training  tape  with  target  launch  weapon  capabilities. 

#78:  1.)  Tape  drives  changed  to  3  1/2"  disk  drives. 

2.)  Different  easier-to-use  fasteners  used  to  secure  equipment 
drawers  when  they  are  closed. 

#79:  Eliminate  the  rest  of  the  PMS  CARD  for  SEA  T  and  computer.  Put 

the  instructions  all  on  the  SEAT  or  as  much  as  possible  so  you’re 
not  looking  back  and  forth  at  the  SEA  T  and  PMS  card.  And  more 
detailed,  longer  training  scenarios. 

#80:  I  would  like  to  see  ^APiD  as  a  subprogram  on  the  OP  tape,  along 

with  automatic  systems  communications  test  This  would  be 
more  useful  than  or\the  maintenance  tape. 

#81:  I  suggest  that  the  da  jla  format  be  changed  from  tape  to  floppy 

disk.  This  would  facilitate  less  tapes  and  would  also  decrease 
the  access  time  of  retrieving  and  storing  data.  I  would  also 
recommend  the  reintroduction  of  some  type  of  printing  device  in 
which  all  data  could  be  printed  out  This  would  encourage  people 
to  keep  historical  data  and  conduct  comparisons  during  those  PMS 
actions  where  slight  differences  may  indicate  system  problems. 
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#54;  For  air  target  training  I  would  like  to  be  able  to  select  the  time 
when  a  target  makes  a  showing.  Also  we  should  be  able  to  use 
both  surface  and  air  training  at  the  same  time.  We  should  be  able 
to  track  a  ship  and  have  an  A/C  target  pop  up.  Also  we  should  be 
able  to  select  5  target  profiles  fora  single  system. 

#86:  The  operational  program  In  ROM  to  keep  from  reloading  it  after 

maintenance.  Larger  RAM  for  less  time  reloading  different 
volumes  if  you  need  a  previous  test. 

#87:  I  would  like  to  see  the  rapid  portion  be  more  specific  on  fault 

monitoring. 

QUESTION  #23: 

#5:  The  Phase  II  tapes  are  very  useful  and  instead  of  using  5  tapes  for 

a  PMS  check  it  takes  2  or  3  to  do  them. 

#7:  Still  under  use  for  total  capabilities. 

#8:  Thanks  for  condensing  the  amount  of  tapes  and  adding  the  help 

function. 

#1 1:  Overall,  the  tapes  are  much  easier  to  use  and  makes  maintenance 

time  less. 

#13:  Good  overall,  decreases  workhad. 

#14:  Phase  II  tapes  are  very  "user  friendly”. 

#19:  MRCs  using  SEA  T  Phase  II  tapes  added  danger  tags  not  used  with 

old  tapes.  Bad  idea.  Takes  too  much  time  for  simple  checks. 

#21:  When  doing  inspect  and  change  you  can  only  type  in  a  certain 

amount  of  data  then  a  new  screen  appears.  With  the  old  tape  you 
can  type  in  data  for  days  with  no  changing  of  the  screen. 

#22:  OLM  is  very  useful.  The  efforts  put  in  to  ease  the  Fleet's  job  is 

noticeable.  Definitely  enhances  troubleshooting  the  system. 

#23:  OLM  was  again  the  greatest  idea  yet! 

#24:  Greatly  improves  fault  isolation.  I  use  them  with  great 

confidence. 
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#25;  Remove  the  RAPID  program  from  the  maintenance  tapes.  We  have 
lost  a  lot  of  completion  time  in  the  workcenter  because  one  card 
in  the  SDC  is  on  order. 

#33;  /  like  them.  They  work  alot  better  than  the  old  tapes. 

^35:  Which  buttons  to  push. 


#42;  Overall  helpful. 

#45;  Snapshot  should  be  made  useful  to  technician  via  applicable 
documentation  or  deleted. 

MAS:  Snapshot  should  not  be  available  on  SF program  tapes.  The 

information  is  of  no  real  value  to  the  technician  level  due  to  the 
random  sampling  during  the  OP  PRG  cycle.  Since  we  cannot  ref.  a 
PRO  cycle  printout  to  determine  the  true  values  to  be  seen  In  the 
printout  on  the  screen  of  register  data. 

Director  evaluation  and  troubleshooting  using  the  SEAT  can  be 
very  useful  if  the  test  had  references  to  the  pubs  with  the  signals 
and  functions  under  test.  Then  director  response,  alignment,  and 
overall  operability  could  quickly  be  identified  and  repaired  as 
necessary. 

The  SEA  T  should  have  a  different  display  incorporated  into  it. 

The  plasma  display  is  too  expensive  and  its  life  is  too  short  to 
justify  the  high  cost  in  these  times  of  responsible  spending. 

1f47:  Full  keyboard  to  enhance  troubleshooting  techniques. 

4f48:  I  went  to  school  with  the  old  tapes,  so  at  this  point,  I  stumble 

through  with  the  new  ones.  Tm  sure  that  a  few  months  from  now 
I  will  be  more  proficient  and  much  happier  with  the  results  the 
Phase  II  tapes  will  provide  during  all  phases  of  my  work. 

M53:  The  tapes  are  easier  to  use  than  the  others  due  to  the  fact  that 

they  are  user  friendly.  Makes  PMS  and  troubleshooting  faster  and 
easier. 

if54:  I  have  yet  to  have  the  opportunity  to  accurately  run  training  due 
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to  fault,  but  find  that  the  information  available  is  far  superior  to 
the  old. 

4f55:  Time  consuming  to  load  operational  program  in  order  to  run 

RAPID. 

4164:  Overall  excellent  package.  Easy  to  use  expanded  training  tape 

would  be  great.  Including  any  real  world  threat  profiles  where 
the  trainer  picks  the  weapon  and  then  is  given  options  really 
available  for  employment. 

4f68:  Are  much  better  than  old  tapes. 

4f78:  1.)  Change  tapes  to  3  1/2”  disks  to  reduce  number  of  broken, 

damaged  and  tost  tapes.  Also  to  reduce  storage  area  and  number 
of  tapes. 

2.)  On  inroect  and  change  screen  used  to  change  information,  it 
might  be  more  advantageous  to  give  it  the  ability  to  contain  all 
the  changed  information  on  one  screen  for  those  of  us  who  have 
large  lists  of  adaptation  data  (10  to  15  line  items). 

4f79:  The  tapes  do  not  last  very  long;  they  go  bad  very  quickly. 

4480:  Would  it  be  possible  to  use  higher  grade  tapes  ?  We  have  had 

several  tapes  go  bad  in  the  past.  Due  to  stretching  and  normal 
use  within  a  month. 

4181:  Please  continue  your  ongoing  efforts  to  make  the  programs  more 

"user-friendly”. 

4486:  I  feel  that  Phase  II  is  a  vast  improvement  and  saves  the 

technician  a  large  amount  of  time. 

4487:  We  would  like  to  have  more  training  on  tho  tapes  so  we  know 

exactly  what  we  are  looking  at  on  some  portions.  Suggest 
hands-on  training/classes  of  Phase  II  tapes.  We  have  (USS 
Obannon)  never  been  indoct.  on  Phase  II. 
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